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mization problems a technique recently introduced by Guerra (cond-mat/0205123) to prove that the 
^q' replica method generates variational bounds for disordered systems. We analyze a family of models 

r—{ , that includes the Viana-Bray model, the diluted p-spin model or random XOR-SAT problem, and 

the random K-SAT problem, showing that the replica method provides an improvable scheme to 
obtain lower bounds of the free-energy at all temperatures and of the ground state energy. In the 
case of K-SAT the replica method thus gives upper bounds of the satisfiability threshold. 



£ . I. INTRODUCTION 

P3 ' 

The replica method jl],^| , originally devised as a trick to compute thermodynamical quantities of physical systems 
in presence of quenched disorder, has found applications in the analysis of systems of very different nature, as Neural 
Networks, Combinatorial optimization problems [Q-Q, Error Correction Codes || etc. 

Although many physicists believe that the method, within the Replica Symmetry Breaking scheme of Parisi [|| , is 
able to potentially give the exact solution of any problem treatable as a mean field theory, the necessary mathematical 
foundation of the theory is still lacking, after more then 20 years from its introduction in theoretical physics. The 
*^ . last times have seen a growing interest of the mathematical community in the method, leading to important but still 
partial results, confirming in certain cases the replica analysis, with more conventional and well established techniques 
P| . Apart the remarkable exception of the analysis of the fully connected p-spin model in ref. Q and the rigorous 
analysis of Random Energy Models |?J , the analysis of the mathematicians has been, as far as we know, restricted to 
the high temperature regions and/or to problem of replica symmetric nature. 

Very welcomed have been the techniques recently introduced by Guerra and Toninelli j^] which allow rigorous 
analysis not relying on the assumption of high temperature, and valid even in problems with replica symmetry 
breaking. Along these lines, an important step towards the rigorous comprehension of the replica method, has 
been undertaken in ||, where it has been shown how in the case of the Sherrington-Kirkpatrick model, and its p- 
spin generalizations, the replica free-energies with arbitrary number of replica symmetry breaking steps constitute 
variational lower bounds to the true free-energy of the model. As stated in that paper, the analysis is restricted 
to fully-connected models, whose replica mean field theory can be formulated in terms of a single n x n matrix. 
However, in recent times, many of the more interesting problems analyzed with replica theory pertain to the so called 
"diluted models" where each degree of freedom interacts with a finite number of neighbors. The introduction of a 
"population dynamics algorithm" ]l0| has allowed to treat in full generality -within statistical precision- complicated 
sets of probabilistic functional equations appearing in the one step symmetry broken framework of diluted models. 
The same algorithm has been used as a starting point of a generalized "belief propagation" algorithm for optimization 
problems pl] , ^2[ . Furthermore, at the analytic level, simplifications due to graph homogeneities in some cases [ [l3|| , 
and to the vanishing temperature limit in some other cases have led to supposedly exact solutions of the ground 
O ■ state properties of diluted models, culminated in the resolution of the random XOR-SAT on uniform graphs in |l5| ] 
and the random K-SAT problem in [Q within the framework of "one-step replica symmetry breaking" (1RSB). 

The aim of this paper, is to show that the replica analysis of diluted models provides lower bounds for the exact 
free-energy density, and ground state energy density. We analyze in detail the cases of the diluted p-spin model on 
the Poissonian degree hyper-graphs also known as random XOR-SAT problem and the random K-SAT problems. We 
expect that along similar lines free-energy lower bounds can be found for many other diluted cases. 

The Guerra method we use sheds some light on the meaning of the replica mean field theory. The physical idea 
behind the method is that within mean field theory one can modify the original Hamiltonian weakening the strength of 
the interaction couplings or removing them partially or totally, and compensate this removal by some auxiliary external 
fields. In disordered systems these fields should be random fields, taken from appropriate probability distributions 
and possibly correlated with the original values of the quenched variables eliminated from the systems. One is then 
led to consider Hamiltonians interpolating between the original model and a pure paramagnet in a random field, 
and by means of these models achieving free-energy lower bounds. We will see that the RS case corresponds to 
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assuming independence between the random fields and the quenched disorder. The Parisi RSB scheme, assumes at 
each breaking level a peculiar kind of correlations, and gives free-energy bounds improving the RS one. 

Our paper is organized in this way: in section 2 we introduce some notations that will be extensively used in the 
following sections. In section 3 we introduce the general strategy to get the replica bounds; we then specialize to the 
replica symmetric and the one step replica symmetry broken bounds, giving the results in the p-spin and the K-SAT 
cases. Conclusions are drawn in section 4. In the appendices some details of the calculations in both the p-spin and 
the if-SAT cases are shown. 

Our results will be issue of explicit calculations. Although at the end we will get bounds, formalizable as mathe- 
matical theorems, the style and most of the notations of the paper will be the ones of theoretical physics. 



II. NOTATIONS 



The spin models we will consider in this work are defined by a collection of N Ising ±1 spins S = {Si, ...,Sn}, 
interacting through Hamiltonians of the kind 

M 

WC«)( S ,J) = 2^0.5(5^,...,%) (1) 

i«=l 

where the indices if are i.i.d. quenched random variables chosen uniformly in {1, N}. We will call each term Hj^ 
a clause. The subscript in the clauses indicates the dependence on a single or a set of quenched random variables, 
as it will be soon clear. The number of clauses M will be taken to be proportional to N. For convenience we will 

choose it to be for each sample a Poissonian number with distribution w(M, aN) — e^ aN ^^p — . The fluctuations of 
M will not affect the free-energy in the thermodynamic limit, and this choice, which slightly simplify the analysis, 
will be equivalent to choosing a fixed value of M equal to aN. The clauses themselves will be random. The p-spin 
model (Ta] has clauses of the form 

HjW •••! <%) = J^Si* ■ ■■■ ■ . (2) 

This form reduces to Hj^){Si», S^) = J^S^S^ in the case of the Viana-Bray spin glass p = 2. In both cases 
the J M will be taken as i.i.d. random variable with regular symmetric distribution ^{J)^ fi(—J). Notice that for 
fi(J) = 1/2[5(J + 1) + 5{J — 1)] the model reduces to the random XOR-SAT problem jl6[ of computer science. The 
random K-SAT clauses have the form JlJ] 

(<%,•••,-%) = JJ ^ — -, (3) 

i=i 

where the = ±1 are i.i.d. with symmetric probability. (The number p of spin appearing in a clause is usually 

called K in the K-SAT problem, for uniformity of notation we will deviate from this convention). Notice that in all 
cases, on average each spin participate to a = ^ clauses, and that the set of spins and interactions defines a random 
diluted hyper-graph of uniform rank p and random local degree with Poissonian statistics in the thermodynamic limit. 
At high enough temperature, the existence of the free-energy in the thermodynamic limit for models of this kind has 
been proved in by Talagrand in fT^ | , together with the validity of the RS solution. A proof valid at all temperature 
based on the ideas presented in this paper, can be obtained for even p in analogy of the analysis in [0 for long range 
models. We sketch it in appendix C in the case of the p-spin model. 

In establishing the free-energy bounds we will need several kind of averages: 

• The Boltzmann-Gibbs average for fixed quenched disorder: given an observable A(S) 

^ {A) = Es^(s)cx P HW(s,j)) (4) 
z 

where Z — exp(— /37i(S, J)) and (3 is the inverse temperature. 

Obviously, oj(A), as well as Z will be functions of the quenched variables, the size of the system and the 
temperature. This dependence will be made explicit only when needed. 
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• The disorder average: given an observable quantity B dependent on the quenched variables appearing in the 
Hamiltonian, we will denote as E(B) its average. This will include the average with respect to the J variables 
and the choice of the random indices in the clauses as well as with respect to other quenched variables to be 
introduced later. 

• We will need in several occasion the "replica measure" 

n(A 1 ,...,A n ) = E(co(A 1 )...co(A n )) (5) 



and some generalizations that we will specify later. 

• We will occasionally use other kinds of averages, as well as other notations, for which we will use an angular 
bracket notation, with a subscript indicating the variable(s) over which the average is performed, e.g. an average 
over a random variable u with probability distribution Q{u) will be denoted equivalently as J duQ{u){-) = 
j dQ(u)(-) = (•)„. Analogously, averages over distribution families of Q(u) will be denoted as J dQQ(Q)(-) = 
J T^Q{Q){-) = (-)q- Subscripts will be omitted whenever confusion is not possible. 

• Another notation we will have the occasion to use in the one for the overlaps among I spin configurations 
{Sf 1 , Si 1 }, out of a population of n {Sj , S"™}: 

1 N 

q(ai ,..., ai) = Y^S? 1 •■■■■S? 1 (l<«r<n Vr), (6) 



and in particular 



1 N 

i=l 



This notation will be extended to multi-overlaps in the 1RSB case, as we will specify in section [II B 



In the following we will need to consider averages where some of the variables are excluded, e.g. the averages when 
a variable is erased. These average will be denoted with a subscript — e.g. if an uj average is concerned the 
notation will be w(-) ■ . Other notations will be defined later in the text whenever needed. 

Our interest will be confined to bounds to the free-energy density fjy = — -^ElogZ and the ground state energy 
density Uqs — limjv— «x> l/NE [min (Un)] valid in the thermodynamic limit, so that 0(1/N) will be often implicitly 
neglected in our calculations. 



III. THE GENERAL STRATEGY 



The strategy to get the replica bound is a generalization of the one introduced by Guerra in the case of fully 
connected models We will consider models which will interpolate between the original ones we want to analyze 
and pure paramagnet in random fields with suitably chosen distribution. The underlying idea is that, given the mean 
field nature of the models involved, if one was able to reconstruct the real local fields acting on a given spin variable 
via a given hyper-edge, and to introduce auxiliary fields acting on that variable in such a way to energetically balance 
the deletion of the hyper-edge, then it would be possible to have an exact expression for the free-energy in terms 
of such auxiliary fields even when the whole edge set was emptied. However, if the replacement is done with some 
approximate form of the auxiliary fields distribution function, the real free-energy will be the one calculated using the 
approximate fields plus an excess term at every step of the graph deletion process. The proof of the definite sign of 
this excess term gives a way to determine bounds for the thermodynamic quantities. 

We will prove the existence of replica lower bounds to the free-energy density of the p-spin model and the random 
K-SAT problem. In this last case our result proves that the recent replica solution of Q gives a lower bound to the 
ground state energy and therefore an upper bound for the satisfiability threshold. The proofs will strictly hold in the 
N — > oo limit, due to the presence of corrections of order 1/N in the calculated expressions for any finite size graph. 
Moreover, our proofs will be restricted to the p-spin model the the K-SAT with even p. In the cases of odd p the 
same bound would hold if one could rely on some physically reasonable assumptions on the overlap distribution (see 
below) . 
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Our analysis will start from the TAP equations for the models [[18 10 1, and their probabilistic solutions implied by 
the cavity, or equivalently the replica method at various degrees of approximation. We will consider in particular 
the replica symmetric (RS) and one step replica symmetry broken solutions, but it should be clear from our analysis 
how to generalize to more steps of replica symmetry breaking. In the TAP/cavity equations one singles out the 

contribution of the clauses and the sites to the free-energy and defines cavity fields h$ and i^, respectively as the 
local field acting on the spin i in absence of the clause fj, and the local field acting on i due to the presence of the 
clause /i only. If we define Z^iSi] as the partition function of a given sample with N spins where all but the spin i 
are integrated, i*V _ j the free-energy of the corresponding systems where the spin Si and all the clauses to which it 
belongs are removed, we can write, 

fiGTi S-[i,...,S-n 

2 p 

= eH"*.-' J] BWeK 5 * (8) 

where Tj is the set of clauses containing the spin i, and the constant B$ = e~@ AF v ' can be interpreted as suitable 
shifts in the free-energy due to the contribution of the clause /i for fixed value of the spin i. We notice that denoting 
J p as J, and renaming the fields in (^) into hi, hp-i, Eq. (||) defines functions 

uj(hi,...,h p -i) and Bj(hi, hp-i) . (9) 

The equation are closed by the self-consistent condition: 

h^= Yl 4° (io) 

These equations are at the basis of iterative algorithms such as the "belief propagation" or "sum-product" know for 



a long time in statistical inference |L9|] and coding theory |20 and the more recently proposed algorithm of "survey 
propagation" Conditions (||) and ( [To|) can be diagrammatically represented as in fig.(Q). 
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FIG. 1. Diagrammatic representation of the cavity relations for h and u fields acting on spin 5*. The hyper-edge interaction 
is drawn in the factor-graph notation. 



The cavity fields solutions of (|jl0) are random variables which fluctuate for two reasons [p|, ^0|j2l| l . First, they differ 
from sample to sample. Second, within the same sample the equations can have several solutions which can level-cross. 
The cavity/replica method provides under certain assumption probabilistic solutions. In the RS approximation, one 
just supposes a single solution to give the relevant contribution in a given sample. The sample to sample fluctuation 
induce probability distributions P(h) and Q{u) whose relations implied by (|^,[l0|) are: 
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P(h) = 53 e~ ap ^f- I du i Q(ui)...du k Q(u k )S(h - ^ u k ) (11) 
k ' i=l 

Q(u) = J dht P{h 1 )...dh p _ 1 P{h p - X ){5{u - uj(h!, hp-{))) j (12) 

where (-)j denotes the average over the random variables appearing in a clause. In addition to sample to sample 
fluctuations, the 1RSB solution assumes fluctuations of the fields from solution to solution of the equations, so 
that the functions P(h) and Q(u) will be themselves randomly distributed according to some functional probability 
distributions V(P) and Q(Q) related by the self-consistency equations p2f 

Q(Q) = J VP 1 V(P 1 )...VP p - 1 V(Pp-x)(S(Q(-)-Q(-\P 1 ,...,P p -i,J))) J (13) 
p(P)=53e-«pl^L mVQ&mSiPi-) - P(-\Q u ...,Q k )) (14) 

7 n ' " 1 — i 



fc=0 1 = 1 



where: 



Q(u\P 1 ,...,Pp- 1 ,H)=N P [P 1 ,...,P p - 1 ] J dhi Pi(/ii)..i/i p -i P 1 (Vi)Bj(fti,..,Vi) m %-«j('«i,..,Vi)) (15) 
P{h\Q 1 ,...,Q k )=M Q , k {Q 1 ,...,Q k ] (2cosh(Ph)) m Jf[d Ul ^—^L^-S(h-'pui) (16) 

where Afc} tk [Qi, Qfe] and A/"g[Gi, G p -i] insure normalization and -Bj(gi, ...,g p _i) is a rescaling term of the form 
(^) that can be reabsorbed in the normalization in the case of the p-spin model. Its form for the K-SAT case is 
given in the appendix, m is a number in the interval (0, 1], which within the formalism selects families of solutions at 
different free-energy levels. The physical free-energy is estimated maximizing over m. 

The interpretation of these equations has been discussed many times in the literature p 10 2lJ]. We will show here, 



that such choices in the field distributions result in lower bounds for the free-energy analogous to the ones first proved 
by Guerra in fully connected models. In order to prove these bounds, we will have to consider auxiliary models where 
the number of clauses aN will be reduced to atN (0 < t < 1), while this reduction will be compensated in average 
by some external field terms of the kind: 

fei 

i h = l 

where the numbers fcj will be i.i.d. Poissonian variables with average ap(l — t). As the notation suggests, the fields u\ 
will play the role of the cavity fields u$ of the TAP approach, and they will be i.i.d. random variables with suitable 
distribution. Indeed, for each field we will chose in an independent way p—1 primary fields g l ^' n (n = 1, ...,p — 1) 
and clause variables j'" n such that the relation 

u%=u J . i .»$> 1 ...£*- 1 ) (18) 

is verified. Notice that the compound Hamiltonian 

wiS[S]=W (at) [S]+?^t[S] (19) 

will constitute a sample with the original distribution for t — 1, while it will consist in a system of non interacting 
spins for t = 0. The key step of the procedure, < 
will also find useful to define fields hi verifying 



spins for t = 0. The key step of the procedure, consists in the choice of the distribution of the primary fields g 1 ^ . We 



hi = J2 u i- ( 20 ) 
i=i 
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The field u are related to the g's by a relation similar to (||), while the h's are related to the u's by a relation similar to 
(|Io|). Of course, the statistics of the fields h and the g's do coincide in the TAP approach. It is interesting to note that 
the bounds we will get, are optimized precisely when their statistical ensemble coincide. As we mentioned, various 
Replica bounds are obtained assuming for the fields g.* the type of statistics implied by the different replica solution. 
So, the Replica Symmetric bound is got just supposing the field as quenched variables completely independent of 
the quenched disorder and with distribution G(g). For the one-step RSB bound on the other hand the distribution 
G will itself be considered as random, subject to a functional probability distribution Q[G]. More complicated RSB 
estimates, not considered in this paper, can be obtained along the same lines. The case of the fully connected models 
considered by Guerra can be formalized in this way where the various field distributions involved are Gaussian. 

A. The RS bound 

We consider in this case i.i.d. fields u and h distributed according probabilities Q(u) and P(h) verifying the following 
relation with the distribution Q(g) of the primary fields. 



Q(u) = J dgi G(5i)...dg p _i G{g p - X ){5(u - Uj(gi, g p -i))) j (21) 
P(h) = P(h\k)Tr(k, ap(l - t)) (22) 

P(h\k) = J dm Q{u x )...du k Q(u k )S(h - «*) (23) 

i— 1 

The distribution G(g) will be chosen to be symmetric under change of sign of g, and regular enough for all the 
expression below to make sense. The RS bound can now be obtained following a procedure to the one of Guerra for 
the SK model, and considering the t dependent free-energy; with obvious notation: 

F{t) = Jim F N (t) = lim --±-ElogZ N (t) (24) 

N^oo N— >oo ply 

where E represents the average over all the quenched variables, the one defining the clauses and the external fields. 
We then consider the t derivative of Fjv 

As in H we will then write 

F(l)=F(0)+j\tj t F(t) (26) 

and show, by an explicit computation, that, up to 0(1/ N) terms that will be systematically neglected, the expression 
coincides with the variational RS free-energy plus a remainder. In fortunate cases this term will have negative sign 
and neglecting it will immediately result in a lower bound for the free-energy. This happens in the Viana-Bray model, 
the p-spin and the K-SAT for even p. In the cases of odd p we were not able to prove the sign definiteness of the 
remainder, although as we will discuss we believe this to be the case on a physical basis. 

The time derivative of F take contributions from the derivative of the distribution of the number of clauses M 

rf7r(M, atN) = _ N << M ^ aW ^ _ * M _ l ab N)) (27) 
dt 

and the distribution of the number of u fields on each site 

dir(ki, ap(l — t)) 



dt 



= ap(ir(ki,ap(l - t)) - 7r(fc 4 - 1, ap(l - t))) (28) 



so that: 



^-E\ogZ(t) = -Na^2(n{M,atN) - tt(M - 1, atN))E' log Z(t) 



dt 



G 



where we have denoted as E' the average with respect to all the quenched variables except M and with E" the average 
with respect to all the quenched variables except ki, and simply Z(t) the partition function of the N spin system 
Z N {t). 

In the first term of (|29|) we can single out the M-th clause, and write Z(t) = Z-M{t)oj(c ^ M ' *p')-m, 



where by Z-j^if) we denote the partition function of the system in absence of the M-th clause, and w(-)_m is 
the canonical average in absence of the M-th clause. In the following terms we single out the fc^-th field u term, 
Z(t) = Z_ k t (t)io{e^ u i ' Si ) ^ , where Z_ ^ (t) is the partition function in absence of the field —it** and analogously 
for the average u;(-) ^ . Finally, rearranging all terms we find 

j t ElogZ(t) = Na^2(n(M - ^atN^E'logMe' "^^'-'^)^)] 

M 



-pa 



EE 7 ^* - hap(l-t))El'\ogMe^ s >)^}. (30) 



i ki 



where we have used X^7r(M — 1, atN)E' log Z_m = ^2 k . ir(ki — l,ap(l — t))E" \ogZ_ = ElogZ. We notice at 

this point that the statistical ensemble defined by 7r(M — 1, atN))E' can be substituted with the original one E and 
the average of the variables appearing in the clause we have singled out. To be more precise, we remark that the 
average o>(-) depends on the quenched variables D — {J,u} appearing in the Hamiltonian. Writing explicitly this 
dependence as uj(-\D), and denoting as D—m all the quenched variables except the ones appearing in the M-th clause, 
our statement is that thanks to the Poissonian distribution of M and the uniform choice of the indices of each clause, 

J>(M - l,atAO)B'log[ W (e- /,H ^ ( V--V ) |D_ Jir )] = E I -L £ (log[a;(e-^^,-.^)| jD) ] )j j . (31) 

M y ii,...,i p J 

where by (■),/ we denote the average with respect to the random variables appearing in the clause. This is a crucial 
step in our analysis, in fact, similar considerations apply to the term in the second line of (30), which can be written 
as 

5>(fc,: - l,op(l - t))E" logKe^'L^] = Eilogu {e^)) u . (32) 

ki 

The same kind of averages E and u> appear in the two terms which can be therefore directly compared as we will do 
in the next section. This property, linked to the Poissonian character of the graph defined by the model would not 
hold for other ensembles of random graphs and the analysis would be technically more involved. Substituting in ( |30| ) 
we find: 



— —E\ogZ(t) = aE 
Ndt B W 



± E (log^(e-^'- A ^)])H-|E( lo S^( e ^)) U 

%x,...,ip 



(33) 



Rearranging terms and using ( ^6|) we finally find that the free-energy f jy can be written as 

F N - F var [G] + f dt Rus [G, t] + 0(1/ N) (34) 
Jo 

where F var [G] coincides the expression of the variational free-energy in the replica treatment under condition G[h] — 
P[h] V h at t = and /„ \ dt R RS [G,t] is a remainder term. Instead of writing the formulae for general clauses, in 
order to keep the notations within reasonable simplicity, we specialize now to the specific cases of the p-spin model 
and the K-SAT. Notice that in all models 

F[0] = -i(log(2cosh(/3/t))) h | t=0 (35) 
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1. p-spin 



In the case of the p-spin Hj(Si 1 , Si p ) — J ■ ... • Si p . Substituting in eq.(|33J) and rearranging terms one 
immediately finds: 

F v p - spm [G] = i [a(p(log(cosh/3n)) u - (log (cosh /3 J)) j) - (log(2cosh/3/ l )) /i + 



(p- 1) / log ( 1 + tanhOSJ) JJtanhC9 5t ) j \ 



(36) 



while the remainder is the t integral of 



R P R S spin [G,t] = -~ 



J2 E < 1 °S( 1 + tanh(/3JV(S u ...£,,))>, - pi? (log(l + tanh^M^))), 



Np 



[p - l)E I log(l + tanh(/3J) JJ tanh(/? 5p )) 



{9t},J. 



(37) 



The expression for F p ~ r spm [G] coincides with the RS free energy once extremized over the variational space of prob- 
ability distributions G [p3[ . Terms have been properly added and subtracted in order to get a remainder which equal 
to zero if maximization over G is taken, and the temperature is high enough for replica symmetry to be exact |jl7|| . As 
we will see, the remainder turns out to be positive. F p ~ r spin [G] is therefore, for all G for which its expression makes 
sense, a lower bound to the free-energy. At saturation the condition 

G[h] = P[h]\ t = Vh (38) 

should hold, which is simply the self-consistency RS equation. 
By using equation 

E (log(l + tanh(/3u)w(S , . i ))) u = E Aog(l + tanh(/3J) \\ tanh(j3g r )u(Si))\ (39) 

\ 4=1 / {g t },J 

we can establish that the remainder is positive for even p. We expand the logarithm of the three terms in (absolutely 
converging) series of tanh(/?J), and notice that thanks to the parity of the J and the g distributions, they will just 
involve negative terms. We can then take the expected value of each terms and write 

1 00 1 r i 

R p RS spm [G, t] = - ]T<tanh 2 " /3J)j±(l [(g<*0)* - pq^ (tanh 2 " fa)^ + {p- l)(tanh 2 " f3g} p \ (40) 



n=0 



where we have introduced the overlap and the replica measure Q defined in section 2. The series in ( |40| ) is an 
average of positive terms in the case of the Viana-Bray model p = 2, where we get perfect squares, and more in general 
for all even p, as we can easily, starting from the observation that in this case x p — pxy v ~ x + (p — l)y p is positive or 
zero for all x = q^ ,y = (tanh 2 ™ f3J)j real. 

In the case of p odd, the same term is positive only if x is itself positive or zero. The bound of the free-energy would 
therefore be established if we were able to prove that the probability distributions of the g( 2n ) has support on the 
positives.^] This property, which tells that anti-correlated states are not possible, is physically very sound whenever 
the Hamiltonian is not symmetric under change of sign of all spins. In fact, one expects the probability of negative 
values of the overlaps to be exponentially small in the size of the system for large N . Unfortunately however we have 
not been able to prove this property in full generality. Notice that upon maximization on G, the results of [ fl7| imply 
that the remainder is exactly equal to zero if the temperature is high enough for replica symmetry to hold. 



1 A different sufficient condition for the series to have positive terms is that |f2(q' 2n ')| > (tanh(/3g) 2n ) g , but it is not clear its 
physical meaning. 
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2. K-SAT 



In the case of the i^-SAT, using def.(0) for the clause H, we find relation: 



uj{hi, hp-i) = uj({J t }, {h t }) = ^ tarda 1 



1 rr p_1 1 1+Jt tanh (0M 

2 1 lt=l 



1 + | 



n^ 1 (i±i 



tanh(/3/it 



where £ = e ^ — 1 < 0. Via direct inspection, the variational free-energy coincides with the RS expression (lj 
1 



F, 



K-SAT 



G 



«( P -i)(io g fi +(e -»-i,n(i±i^))\ 

Mlog(2 coMM))u - ap /log ( 1 + J] ^ + tanhC/fc) 



(log(2cosh(/3/i))) h - 



{fft},{^}. 



while the remainder is the t integral of 



nK-SAT 
^RS 



^ E (togfl + (e-"-lMn 

ii,...,ip \ \ t=l 



^ 1 + JtS U 



+ 



{gt},J,Ut} 



(p-i)(io g (l+eil 



1 + J r tanh(/3g t ) 



{9t},{Jt}. 



(41) 



(42) 



(43) 



Considerations analogous to the case of the p-spin, have led us to add and subtract terms from eq. (|33J) to single out the 
proper remainder term. Expanding in series the logarithms, exploiting the symmetry of the probabilities distribution 
functions and taking the expectation of each term of the absolutely convergent scries we finally obtain: 



R 



K-SAT 
RS 



ci ( — 1 ) n r 

= E —^(O n n (1 + QnY -p(l + Q„)((l + Jtanh(^))")^ 1 + (p - !)<(! + Jtanh(/3 ff ))")^ 



^ n>l 



(44) 



where we have defined f* = £/(2*>) < and Q n = £" =1 (J')j T,aT<...<a, <7 ai " a! - Detailed calculations are given in the 
appendix. As in the p-spin case, the previous sum is obviously positive for p even. For p odd we should again rely 
on the physical wisdom that all q( ai >---> ai > have positive support and so have the functions 1 + Q n > 0. Again, the 
variational free-energy coincides with the RS expression once extremized over G at the condition P — G at t = 0. 



B. The 1RSB Bound 



We establish here a more complex estimate, in a larger variational space of functional probability distributions. 
The general strategy will be here to consider the same form for the auxiliary Hamiltonian, but now with a more 
involved choice for the fields distribution. The fields on different sites or different index U will be still independent, 
but each site field distribution G^{g^) will be itself random i.i.d., chosen with a probability density functional Q[G], 
with support on symmetric distributions G(—g) — G(g). It will be assumed that Q is such that all the expressions 
below make sense. In this case, the variational approximation for the free-energy will be obtained from an estimate of 

-pF N [m,t} = ^-E 1 \ogE 2 (Z m (t)) (45) 
mN 

where we have denoted with: 
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E 2 the average w.r.t. <jr|*'" for fixed distributions G l ?' n according to the measure 



N ki p-1 
i=l = i n=l 



2 cosh(/3w jh ,„ ' 1 . . .5'* 1 ) ) 



(46) 



where C ensures the normalization. 

• E\ the average with respect to the quenched clause variable, distributions the G 1 / 's and the Poissonian variables 
ki's, which will be i.i.d. with probabilities /i(</), Q{G 1 ^) and ir(ki 1 (1 — t)a) respectively. 

The number m is real in the interval (0,1]. The statistical ensemble of the auxiliary fields u and h will be now related 
to the one of the g by: 



Q(Q) = J VG 1 g(G 1 )...VG p - 1 g(G p - 1 ){5(Q(-)-Q(-\G 1 ,...,G p - 1 ,J)))^ 



(47) 



P(P) = f>- t))k f WVQtQiQtfiPi-) - P{-\Q u ...,Q k )) 
k=a " ' J i=i 



(48) 



where: 



Q(u\Gi, Gp-i, J) = jV G [Gi,...,G p _i] J d gi Gi(gi)...dg p _i G 1 (g p - 1 )B J (g 1 , ffp _i) m <5(u - uj(g ly (49) 
G(5|Qi,...,Q fc )=^,fc[Qi.-.Qfc] (2cosh(/3 5 )r /n du 



Qi(ui) 



11 (2cosh(/3 U; ))' 



where A/q^Qi, Qfc], 7Vg[Gi, G p _i] and Bj(g\, g p _i) have been previously defined. With notations similar 
to the ones of the RS case, we can write 



d_ 

lit 



{-(3F N [m,t]) = -a^{-K{M,atN) - w(M - 1, at N))E[ j^- log E 2 Z{t) 



M 



+ f EE(^(^ ft P(l - *)) - <h 1, ap(l t)))E'l t ^-\ogE 2 Z(t) 



(51) 



i ki 



which, extracting explicitly the contribution from the M-th close in the first term and the fc^-th field u in the second, 
following considerations similar to the RS case we find: 



d_ 

di 



M 



(-f3F N [m, t] ) = a ^(tt(M - 1, atN) )^E[ log 



s 2 z 



— U 4 U i 



(52) 



Again it can be recognized that the primed averages coincide with the averages over the original ensembles plus the 
averages on the variables appearing in the terms we extracted. Finally we get: 



±(-PF N [m,t]) = ^E 1 



Np Z^i \ ° g 1 E 2 Z m 



P ^2 / lQg ( E 2 Z m (cu(e^) m ) l 



N 



E 2 Z" 



Q 
(53) 



Rearranging all terms one finds the estimate: 
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F N = F var [Q] + [ dt R 1RSB [G,t]+0(1/N) 
Jo 



(54) 



where this time F var [Q] coincides with Firsb[G], the expression of the variational free-energy in the 1RSB treatment 
at the saddle point Q = V at t = 0, and J Q dt R\rsb[G it] is the remainder. Notice that the derivation immediately 
suggests how to generalize the analysis to more steps of replica symmetry breaking. Let us now specialize the formulae 
for the p-spin model and the K-SAT. Again, in this case we will need the expression for F[Q]: 



F[0] 



/3m 



log 



2cosh(/3/i) 



hi pi \ t= 



(55) 



1. p-spin 

In this case, plugging def.(0) in eq. (p3|) rearranging, adding and subtracting terms one finds: 



FLr SPm [G] 



(3m 



log 



am (log(2 cosh(/3 J))) , — ap ( log 



2cosh((3h)J I J p bv yr " IJ *\ b \V2cosh(/3 U ) / 



Gi ,. . ..G P ;J 



+a(p - 1) ^log ((1 + tanh(/?J) tanh(/3 ffl )... tanh(/? ffp )) m ) gij gp 
while the remainder is the t integral of 

J_ ^ / fE 2 Z m (l+Lu(S ll ...S lp )temh((3J)) 
(VP ^ \ og 



(56) 



r^p— spin 
U 1RSB 



[g,t] = -—E 1 



E?Z r 



N 



E 2 Z m ({1 + Lu{S l )tanh(l3u)) m } u 
E 2 Z m 



(p-l) (log((l + tanh(/3J) tanh(/3 ff i)... tanh(/3 ffp )) m ) 



G± , — G V ;J 



(57) 



The expression for FP ar spm [Q] coincides with the 1RSB free-energy [£3| once maximized over the variational space of 
probability distribution functionals Q. The maximization condition reads: 



g[p] = v[p\ | t=0 v p 



(58) 



which is simply the self consistency 1RSB condition. For even p (and in particular for p = 2 that corresponds to the 
Viana-Bray case), one can check that the remainder is positive just expanding the logarithm in series and exploiting 
the parity of the J and the g distributions. As this is considerably more involved then in the RS case, we relegate 
this check to appendix A. 



2. K-SAT 



In the if-SAT case the expression for function BjQii, h v -\) reads: 

B ]ihl , w . B «j t} , {M) = i + 1 n^ 1 + Jt tanh(/3M 



i=l 



(59) 



while the corresponding one for Uj(hi, h p ^i) is the same as in the RS case. The corresponding replica free-energy 
and remainder read 



7 K-SAT 



l 

ap ( log 



1, /! + J t tanh((3g t ) 



(p-l)^log/^l + ff[ 

B({J t },{gt}) 

2co S h(f3u J ({J t },{g t }))J / {9t} / {GtUJt} 



iSt}' {G t },{J t } 



log 



1 



2 cosh(/3/i) 



hi p 



(60) 
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The remainder is the t integral of 



a 

/3m 



{J t } 



sen 



( E 2 z m ((1 + e 1+J ^ (Sl) UtZl 



V 




{G t },{J t },J 



(p-iwiogMi+en 



1 + J t tanh(/? fft ) 




{G t },{J t }. 



(61) 



, with the 
for even p 



The expression for Ff a - SAT [g] coincides with the 1RSB free energy once extremized under condition (J5I 
corresponding K-SAT probability distribution functionals. Notice that The proof of the positivity of (pi 
is again dove via series expansion, all the detail are explained in Appendix B. 

At this point we can take the zero temperature limit, finding that the resulting expression gives us a lower bound 
for the ground-state energy of the system, i.e. the minimal number of unsatisfied clauses. Notice that the T — > limit 
of the replica free-energy is not trivial. The necessary assumptions on the field distributions to get it correct are well 
known in the physical literature, and have been recently reviewed in pi] ]. Recently Mezard, Parisi and Zecchina |l2] ] 
have worked out the K-SAT 1RSB solution for p = 3 predicting a non zero ground-state energy for values of a above 
a satisfiability threshold of a c = 4.256, very well in agreement with the numerical simulations. Our results, together 
with the additional hypothesis of positivity of the support of the overlap functions imply that this value is an upper 
bound to the true threshold. 



IV. SUMMARY AND CONCLUSIONS 



In this paper we have established that the free-energy of some families of diluted random spin models can be written 
as the sum of a term identical to the ones got in the cavity /replica plus an error term. Both the replica term and the 
remainder are different in different replica scheme, corresponding to the choice of statistical ensemble of the cavity 
fields. We believe that the sign of the remainder is in general negative in the model we have considered, although we 
have been able to prove that only in the case of even p. For odd p our belief is supported by the physical wisdom 
that the overlap distributions are supported on the positives in the large N limit. 

We have considered the cases of replica symmetry and one step of replica symmetry breaking. It is clear that the 
analysis could be extended to further levels of replica symmetry breaking, although the complexity of the analysis 
would greatly increase. The 1RSB level is thought to give the exact scheme to treat the p spin model and the K-SAT 
problem for p > 3. For the Viana-Bray model on the other hand it is believed that no finite RSB scheme furnish the 
exact solution, and one needs to consider the limit of infinite number of replica symmetry breaking. It is not clear to 
us how to generalize the analysis to this case. 

Our analysis of the diluted models underlines a strong link between the Guerra method and the cavity method 
which remained rather hidden in the fully connected case. In the cavity approach one considers incomplete graphs in 
which either sites or clauses are removed from the complete graph. Then, with the aid of precise physical hypothesis, 
consistency equations are written that allow to compute the free-energy from the comparison between the site and 
clause contributions. In the approach presented in this paper the removal of clauses is compensated in average by the 
addition of some external fields which have precisely the statistics which is assumed with cavity. The novelty of the 
approach is that it gives some control on the approximation involved, and proves the variational nature of the replica 
free-energies. Of course a complete control on the remainder in various situations would result in rigorous solutions. 

Although in this paper we have mainly worked at finite temperature, the zero temperature limit can be considered 
without harm. This is particularly relevant in random satisfiability problem, where it is typically found a SAT-UNSAT 
transition where the ground state energy passes from zero to non zero values. Our analysis shows that, the replica 
estimates for many of the models considered in the literature, and possibly some of the ones to be obtained in the 
future with the same method provide upper bounds for the satisfiability thresholds. 

In this paper we have confined ourselves to spin models on graphs with Poissonian connectivity. The extension to 
more general diluted graph models will be presented in a forthcoming paper. 
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Finally we would like to remark that despite the heavy formalism, our proofs to the bounds are conceptually simple. 
They are issue of explicit computations and elementary positivity consideration. We hope that this contributes to 
illustrate the elegance of the construction first introduced in ||. 
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APPENDIX A: P-SPIN 



1. Check of the positive sign of Ri R t P g n 



In this appendix we will explicitly show that expression ( p7|) has positive sign. We proceed expanding in absolutely 
convergent series each of the three addend in (p^) and showing, taking the expectation of each term that the resulting 
series is positive semidefinite. 

The first term writes: 



E\ { log 

^ 1 ~ht7! n ri =i ( r i - m)J ... [^-ft n,,=i in -m)){ (tanh(/3 J)^.=i ~ 

P (-rrl E 2 (Z m u(S n ...S tp ) k -) 

A i I iu — — 



Z^i>l l Z-/k lt ...,ki 



(Al) 



where the term -Ei(-) in the last line of eq.(Al) can be written as 

a l,l a k,,l ah.l 



(E 2 z™y 



(q 



(k 1 ....,k i )\p 



(A2) 



where each cj/ s ) (s = 1, I) is a product of k s Gibbs measure with independent fields (variables appearing in the E^ 
averages), and same fields distributions and quenched disorder (variables appearing in Ei). The quantities q( kl >— > fc i) 
have been defined as: 



(fci,...,k { ) 



— _ s 1 ' 1 ■ ■ S k1,1 ■ ■ S 1L ■ ■ S k '' 1 



(A3) 



and in this case the averages are performed using a a generalized replica measure, defined as: 



n l „=i4 B) z™ ) u (a) (s il ...s i j k - 

(E 2 Z m ) 1 



(A4) 



for any integer n. The average over J selects the terms with even E s =i ^ m (Al) so that we finally find 
; 1,00 i / T~[k B — l / 



/>1 



Et e n 

k 1 ,-..,k l s = l 

EL — 



^(tanh(/3J))Sl=i fcs ^ flW J( ? Cfci.-.fcO)pj (A5) 



notice that (r s — m) > V integer r s > only in the current hypothesis that m £ [0, 1]. Analogously, the term 

E 2 Z m ((1 + tanh(/3 U )w(^)) m ) t 



Ei log : 



(A6) 



writes 
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Et e ri(- n - <r - m) 



i 

1>1 s=l * 

or, making use of the definition of G(g), 

l.oo / 

Ey E n 



(A7) 



I ^ / ]lr=l 1 ( r_m ) 



p-1 



n<(tanh(/3 ff )) fe =) ) ((tanh(/JJ))£.=iM n« [( ? ^— fc 0)l (A8) 



Eventually, following analogous manipulations, the last term 

(log( (l + tanh(/3J)ntanh(/3 5t )) \ \ 

' ' {at}' j,{ Gt } 



(=i 



can be written as 



l l,oo 1 / Tl k.-x 



Et E II 

Z>1 fel , - ,*!( s=l 

EL — 



n r l i (r-m) 



n <(tanh(/3 5 ))^) \ ((tanh(^J))X)i=i*- 



Invoking [l?] and collecting all 



i 1,00 1 /■n k ^ 1 f \ s 

« n ^i4Et E II - =1 " m 



8m ^— ' Z 

^ ;>i fci *i s=i 



(tanh(/3J)) 



El =1 fc * 



EL- 



where we have defined: 



Jjd) r( ? (*l,...,*«))P _ pA ( fclj fc,)P-l(g(fel.-.fc|)) + (p _ 1)^, fc,)J* 



A(ki,...,ki) = (i[((taDh(fig)) k ') s 



(A9) 



(A10) 



(All) 



(A12) 



Each inner term of the series ( All ) 



(A13) 



is always positive semidefinite for p eve n whi le we need the condition conditions q( kl ->~> k > for p odd. For p = 2 one 
retrieves the Viana-Bray result where ( A13 ) is a perfect square. As in the RS case, one can now integrate eq.( All ) 
and recognize that once more the total true free-energy can be written as variational term plus a positive extra one. 
The variational term coincides with the 1RSB free-energy at stationarity and under condition 



G(P)=V(P)\ t = V P 



(A14) 



APPENDIX B: K-SAT 



1. Check of the positive sign of R^ s 

The aim of this appendix is to show that the expression for the remainder Rrs[G, t] in 
case as positive sign. For the if-SAT Rns[G,t] specializes to:n 



J) for the K-SAT model 



The sum of the site indices has been eliminated by symmetry. 
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R^ SAT iG,t] 



-E 



log (w (cxp ^Ilr 



p i+.;,,,s-, 

= 1 2 



{Jt} 



p (log (1 + u{S) tanh(/3 M ))) u - -p ( lo { 



^ + £ ^ l + J f tanh(/3g f ) ^ ~ 



{gt},{Jt} 



(P-1) 



/io g fi + en 1+Jr T h(/3gt) ) 



' / {St}, {■/*}. 



(Bl) 



which thanks to the relation between Q(u) and G(g) 7 rewrites as 



R K ns SAT [GA = ~E 



log 1+ (e^ - l)u,(Q 



1 + JtS t 



\ V V t=l - / / / {9 t }„/,{J t } 



(p-i)(iog(i+en 



1 + J r tanh(/3g t ) 



t=i 



(B2) 



The last term has been added and subtracted from eq. ( p4| ) in order to extract a remainder that would vanish if replica 
symmetry hol ds, a nd maximization is performed on G(g). As in the p-spin case, we will proceed in a Taylor expansion 
of expression (|B2| ) in powers of and rely on absolute convergence to average each term of the series. 
Expanding the first term in (B2) we can write 



(— 1 Y l+1 

£L_i_ (n » E 

n>l 

(— 1 

E — — (n n ^ 
E — — (n nfi 

n>l 



1 + JtSt 



J I {Jt}. 

' P 

H(i+ j t s, 



\t=i 



1,71 



II 1 £ 

t=l \ (=1 ai<...<a ; 
P / n l,7i "\ 

II !+E( J iU E i ai - ai 



,t=i \ ;=i 

(0 B n[(i + Q n ) p ] 



ai<...<a; 



(B3) 



where we have defined C = {e~P - 1)/(2P) and ( J ') j Ea^ 

sign of £*, the coefficients (— l)" +1 (£*) n are all negative. 
The analogous expansion of the second term is: 



Q n . Notice that due to the negative 



1 + JS t-t 1 1 + J t tanh(/3#t) 
9 H 9 



E 



E 



Hog ^1 + & 

(_l)n+l 



(e*) n o 



i+E<A/ E < 

v i=l ai<...<a; 



{J*M{fft}. 
/p-1 n \ 

(nil( 1 + JttsahWat))) 
\t=n=i I {J t },{g t }. 



E (1 + Q„) ((1 + JtanhO^))")^ 



n+l 



n\p— 1 



(B4) 
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Finally, the third terms in eq.(B2) immediately reads 



(lo S (l + m 1 + Jtta ; HP9t) )) = E^(rr((l + ^anh( W )«^ (B5) 

\ V t=l / / {J t },{gt} n>l 

The sum of the three pieces in eq.( |B2| ) gives: 

Rrs SAT \GA = - J2 — ^)" [(! + Qnf -p(l + Q„)<(1 + Jtanh^s))")^ 1 + (p - 1)((1 + Jtanh(/3 5 ))T 

n>l 



(B6) 



The previous sum is always positive semidefinite for p even while we need 1 + Q n > for p odd. 

2. ...and of R^sb T 

We proceed in the same way as in the p-spin case. The algebra is elementary but more tedious and involved, 
therefore we will only list the final results of the calculation. Starting from eq. (|6l|) , we again expand in series the first 
term, getting, with a treatment similar to the RS case: 

i i /v\ k *~ 1 ( — \\ 

«t]=Ey E *' II , " (,) [(i + Q(fex,-, W] (B7) 

1>1 ki,...,kt s=l V s ' / 

where we have defined: 

l ki ,k s s kx,...,k s 

q^, ...,*,) =x; e (^ i+ - +rs) ) j n e (B8) 

s=l n,...,r 3 i— 1 ai<...<a rt — 1 

Analogous steps give for the second term in eq.([6l]) 

Et e (~n^ k ° n ( ukXi )\ ~ m) ) » w [i + Q(fci, *,)] (n (a + jt^Mp g) f)) ( B9 ) 

1>1 ki,...,ki s=l V s '/ \s=l 

and for the third term 

;>i fci,...,fcj s=i \ s ' / \s=i "/ g,j 

where in the last two terms we can further expand 

ll \ ^ / k k I I I \^\ 

i\{{{i+jt^MP9)) ki ) ) = ( e n( fc 0( j(, ' i+ ''' +r,) >7(n« tanh (/35)r) s ) ) (Bn) 

\s=l V G,J Vri,...,n=ls=l ^ s ^ ' \s=l / G/ 

with n equal to p — 1 and p respectively. Since £* < it is easy to see how only positive terms of the series survive. 
Collecting all, we eventually find the complete power expansion for R^r/^'- 



9 1 G,J 



9 / 



(BIO) 



^ Z>1 fci,...,fei 8=1 \ 



X 



[(1 + Q(fci, h)Y - p(l + Q(fc l7 fc,))A(Ai, fcif" 1 + (p - l)A(Ai, fc;) p ] (B12) 
where we have defined 



A(h,...,h) = I H^(l + Jtanh(/3 9 )) fe «) \ (B13) 

\s=l V G 

Again, every term of the expansion is positive for even p and for p odd under condition 1 + Q(fci, ki) > 0. 
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APPENDIX C: APPENDIX C 



Let us briefly sketch the proof of the existence of the thermodynamic limit of free-energy of the p spin model for 
p. Let us define a model which interpolates between two non interacting systems with N\ and iVjj spins respectively, 
and a system of N = iVj + -/V2 spins. Each clause [1=1, ...,M will belong to the total system with probability t, 
to the first subsystem with probability Ni/N(l — t) and to the second subsystem with probability N 2 /N(l — t). We 
chose the indices in the following way: for each clause the indices will be i.i.d. with probability t, the indices 

will be chosen uniformly in the set {1, N}, with probability (1 — t)Ni/N the indices will be chosen in {1, N%} 
and with probability (1 — t)N2/N in the set {Ni + 1, N}. Let us consider the free-energy F]y(t) = -j^-g log Z(t). A 
direct calculation of its i-derivative 



dF N (t) = _1 
dt (3 



N Nf 



Ni+1,N 



ftp Z^i M M? 2-^i ]\f MP 



N N$ 



E(\og(l+taiah(0J)cj(S il ...Si p )))j. 



(CI) 



Expanding the logarithm in series, observing that thanks to the symmetry of the J distribution the odd term vanish, 
introducing the replica measure and using the convexity of the function x p for even p one proves that dF ^ t ^ < 
which implies sub-additivity Fn < ^F^ + -jt-Fn 2 ; this is in turn is a sufficient condition to the existence of the 
free-energy density. The same prove applies to the even p random K-SAT model. For odd p we face a difficulty similar 
to the one in the replica bounds. We can not prove sub-additivity due to the need to consider negative values of the 
overlaps, and non convexity of x p for negative x. 
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